Description

Re-basing #5225 with the latest changes from main.

This PR introduces API Tokens — a new capability in the Security plugin that allows security admins to issue long-lived, scoped tokens and associate permissions directly with the token.

How it works

API Tokens are opaque tokens with the format os_<random>. When a token is created, a SHA-256 hash of the plaintext token is stored in a system index, .opensearch_security_api_tokens. The plaintext token is returned once at creation time and never stored. On each request, the incoming token is hashed and looked up in an in-memory cache populated from the index.

Tokens are authenticated via the Authorization: ApiKey <token> header.

What is novel about this approach compared to OBO tokens is that permissions are scoped directly to the token rather than derived from the issuing user's roles. An admin can issue a token with only the permissions it needs — for example, read-only access to a single index — regardless of the admin's own permissions. This enforces the principle of least privilege and is a key building block toward deprecating Roles Injection, the current practice for how plugins run async jobs with user-scoped permissions.

Revocation model

Tokens use a soft-delete revocation model. When a token is revoked via DELETE /_plugins/_security/api/apitokens/{id}, the document in the index is updated with a revoked_at timestamp rather than being deleted. This means:

• Revoked tokens remain visible in the list endpoint with a revoked_at field, enabling UIs to display revocation history and audit trails.
• Revocation is synchronous — the cache refresh is broadcast to all nodes and confirmed before the response is returned, so the token is immediately unusable cluster-wide.
• During cache reload, tokens with revoked_at set are excluded from the in-memory authentication maps, so they cannot be used to authenticate.

Token Identity & Naming

Token names serve as the user-facing identity. They must be unique and match [a-zA-Z0-9_-]+. In audit logs and internal user contexts, tokens appear as token:<name> (e.g., token:my-service-token). The SHA-256 hash used for authentication lookup is internal-only and never exposed in API responses or audit logs.

System Index Protection

API tokens are denied access to all system indices regardless of their granted permissions. This is enforced at the privilege evaluation layer via SpecialIndexProtection and applies to both legacy and V4 privilege evaluation modes.

API Reference

Create API Token

POST /_plugins/_security/api/apitokens

The create request accepts duration_seconds — how long the token lives, in seconds. The cluster setting max_duration_seconds controls the maximum allowed duration.

Request:

{
  "name": "my-token",
  "cluster_permissions": ["cluster:monitor/health"],
  "index_permissions": [
    {
      "index_pattern": ["logs-*"],
      "allowed_actions": ["indices:data/read/search"]
    }
  ],
  "duration_seconds": 3600
}

Response:

{
  "id": "Nd_pMRWeAC93ZGMhRa5CxX",
  "token": "os_abc123..."
}

The id is used to manage the token, such as listing or revoking it. The plaintext token is returned once and never stored — save it immediately.

List API Tokens

GET /_plugins/_security/api/apitokens

Returns all tokens, including revoked ones. Revoked tokens include a revoked_at field (epoch millis). The expires_at field is the epoch millis timestamp when the token expires.

Response:

[
  {
    "id": "Nd_pMRWeAC93ZGMhRa5CxX",
    "name": "my-token",
    "iat": 1742000000000,
    "expires_at": 1742003600000,
    "cluster_permissions": ["cluster:monitor/health"],
    "index_permissions": [
      {
        "index_pattern": ["logs-*"],
        "allowed_actions": ["indices:data/read/search"]
      }
    ]
  },
  {
    "id": "Xf_qNSZeBC04AHNiSb6DyY",
    "name": "old-token",
    "iat": 1741000000000,
    "expires_at": 1741003600000,
    "revoked_at": 1741500000000,
    "cluster_permissions": ["cluster:monitor/health"],
    "index_permissions": []
  }
]

Revoke API Token

DELETE /_plugins/_security/api/apitokens/{id}

Response:

{
  "message": "Token Nd_pMRWeAC93ZGMhRa5CxX revoked successfully."
}

Revocation is a soft-delete — the token metadata is retained with a revoked_at timestamp. The token is immediately unusable after the response is returned. The cache refresh is broadcast synchronously to all nodes before the response is sent.

Using a Token

Pass the token in the Authorization header using the ApiKey scheme:

Authorization: ApiKey os_abc123...

Example — search a permitted index:

GET /logs-2025/_search
Authorization: ApiKey os_abc123...

Response:

{
  "hits": {
    "total": { "value": 3, "relation": "eq" },
    "hits": [ ... ]
  }
}

Example — attempt a forbidden action:

DELETE /logs-2025
Authorization: ApiKey os_abc123...

Response:

{
  "error": {
    "type": "security_exception",
    "reason": "no permissions for [indices:admin/delete]"
  },
  "status": 403
}

Issues Resolved

Partially resolves #4009, limited to security admins in the initial release.

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff
• V4 (nextgen) privilege evaluation mode supported and tested